Segmentation of images is an important process for many digital image processing techniques. For example, image segmentation may be used for generating depth maps for three dimensional imaging, for object identification, for artificial coloration etc. Efficient interactive image segmentation by an operator is important in e.g. digital photo and image processing or in post-production work for video or digital images. For example the creation of a depth map or disparity map for stereoscopic video requires the specification of object borders in order to assign depth or disparity accurately.
Examples of methods for performing interactive segmentation of images may be found in the paper “GrabCut—Interactive Foreground Extraction using Iterated Graph Cuts” by Carsten Rother, Vladimir Kolmogorov, Andrew Blake, August 2004, SIGGRAPH (Special Interest Group on GRAPHics and Interactive Techniques) 2004 (available at http://research.microsoft.com/vision/Cambridge/papers/siggraph04.pdf).
Currently methods exist for efficient segmentation based on placing so called ‘scribbles’ to roughly identify where an object of interest is located. A scribble is typically a curve manually drawn on an image by a user to specify pixels that are known to belong to a specific object, i.e. which are indicated by the user to belong to the same segment. For example, a scribble may be drawn inside an image object to specify that the underlying pixels are part of the image object. A segmented region may then spread out from the scribble with neighboring pixels being assigned to the same segment until an edge of the object is estimated to be reached (e.g. due to a detection of a colour transition). Alternatively or additionally, a scribble may be drawn which identifies a number of pixels that are indicated by the user to not belong to the specific object and thus which should not be included in the image object segment (typically equivalent to the scribble defining a number of pixels that should be included in a complementary segment to the image object segment). In this case, the same approach may be used to assign pixels to a different segment than the segment corresponding to the image object until a transition is detected. For example, an outside object scribble may be used to define background pixels and an inside object scribble may be used to define foreground pixels for a foreground object. An example of an image with such scribbles is illustrated in FIG. 1.
However, although such approaches may assist in the segmentation of an image, it may provide suboptimal performance for many images. For example, if a colour transition is very small or absent between neighboring objects, the algorithm will often not lead to an optimal and sharp separation between the different segments. Although, this degradation in the segmentation accuracy may be alleviated by placing further and more accurate and detailed scribbles, this substantially increases the required user interaction and results in a cumbersome and laborious process.
Another approach that has been proposed for interactive segmentation is for users to directly define the edges between different objects. For example, a user may trace the specific contour of an image object and the segmentation may use this as a definition of a border between different segments by assigning the pixels on one side of the contour to one segment and the pixels on the other side to another segment. However, this approach tends to be very demanding and time consuming for a user and requires the user to manually trace many (or even all) contours to provide the segmentation. Thus, the segmentation approach is very laborious and cumbersome. Furthermore, the resulting segmentation may often be suboptimal as the user entered contours may not exactly match the underlying object edges. Also, it may be difficult to identify exactly which segments the individual pixels belong in case there is more than one segment represented on one side of a contour. An example of an image with user defined contours is illustrated in FIG. 2.
Hence, an improved approach to image segmentation would be advantageous and in particular a system allowing increased flexibility, reduced complexity, reduced user interaction, improved segmentation and/or improved performance would be advantageous.